In this section we are going to do an explanatory data analysis by using the cleaned data described in the data part. Throughout the section, we will still need some transformation of the data to facilitate the visualization and to understand everything in a deeper way.
To present the crimes and the race in a nicer way, we decide to mutate the former in term of “per 1000 inhabitants” and the latter in percentage terms. This makes sense also because different countries have different dimensions and population size. Therefore, maintaining absolute magnitudes would probably give us a wrong perception and result. We don’t change the variables’ names, though.
As you may have seen in the section on data, we end up having many features. Although some of them might be irrelevant or redundant. To see that we use a straightforward correlation command; this can be already a step towards the selection of most important variables that we may need for our analysis later.
The main findings are:
Moving forward, having observed the correlations above, we can also look into each variable.
To do so we can look up easily at the outcome of the data-set’s summary.
For age:
| Min. | 1st Qu. | Median | Mean | 3rd Qu. | Max. | |
|---|---|---|---|---|---|---|
| Age_0_17 | 0.1676 | 0.2288 | 0.2401 | 0.2394 | 0.2495 | 0.3150 |
| Age_18_24 | 0.0829 | 0.0967 | 0.0999 | 0.1010 | 0.1032 | 0.1446 |
| Age_25_44 | 0.2308 | 0.2542 | 0.2643 | 0.2662 | 0.2761 | 0.3680 |
| Age_45_64 | 0.1144 | 0.2519 | 0.2623 | 0.2609 | 0.2714 | 0.3122 |
| Age_65_84 | 0.0589 | 0.1075 | 0.1151 | 0.1140 | 0.1214 | 0.1609 |
| Age_over85 | 0.0050 | 0.0151 | 0.0174 | 0.0177 | 0.0205 | 0.0269 |
The highest percentage of population is between 25 and 64 years old while the lowest has more than 85 years.
For race:
| Min. | 1st Qu. | Median | Mean | 3rd Qu. | Max. | |
|---|---|---|---|---|---|---|
| White | 0.2557 | 0.7373 | 0.8339 | 0.8020 | 0.8908 | 0.9654 |
| BlackAfricanAmerican | 0.0041 | 0.0326 | 0.0770 | 0.1145 | 0.1566 | 0.5812 |
| Asian | 0.0060 | 0.0138 | 0.0229 | 0.0372 | 0.0407 | 0.4083 |
| Other_race | 0.0122 | 0.0210 | 0.0273 | 0.0463 | 0.0443 | 0.3396 |
The majority of the population is white, followed by black and African-American.
For crimes:
| Min. | 1st Qu. | Median | Mean | 3rd Qu. | Max. | |
|---|---|---|---|---|---|---|
| homicide | 0.0084 | 0.0269 | 0.0454 | 0.0494 | 0.0611 | 0.3487 |
| violent_crime | 0.8655 | 2.6797 | 3.5792 | 4.0563 | 5.0244 | 15.3711 |
| rape_legacy | 0.0972 | 0.2576 | 0.3131 | 0.3271 | 0.3830 | 0.8914 |
| aggravated_assault | 0.5119 | 1.5712 | 2.2706 | 2.5683 | 3.3108 | 8.0413 |
Remember that crimes are expressed in per 1000 terms.
Homicides are the less common crime, while violent crimes and aggravated assault occur on average to 4 and 2.5 people out of 1000.
For mental health expenditure, education, population and GDP:
| Min. | 1st Qu. | Median | Mean | 3rd Qu. | Max. | |
|---|---|---|---|---|---|---|
| mh_exp_pc | 2.423e+01 | 7.144e+01 | 9.883e+01 | 1.201e+02 | 1.451e+02 | 4.099e+02 |
| perc_bscconferred_18_24 | 1.939e+00 | 4.586e+00 | 5.505e+00 | 5.661e+00 | 6.397e+00 | 1.374e+01 |
| perc_bscholder_25_44 | 1.948e+01 | 2.572e+01 | 2.987e+01 | 3.056e+01 | 3.408e+01 | 6.535e+01 |
| Current_dollar_GDP_millions | 2.266e+04 | 7.300e+04 | 1.738e+05 | 5.605e+05 | 3.818e+05 | 1.678e+07 |
| population | 5.091e+05 | 1.715e+06 | 4.352e+06 | 1.173e+07 | 7.092e+06 | 3.160e+08 |
In this last summary table, it’s worth mentioning that
To present the most important data by State we created an interactive map which shows the selected variable distribution in US’s states in a given year.
Moreover, we try to analyze graphically the main variables separately in order to potentially detect outliers or interesting path/characteristics.
We start with a time series for mental health expenditure per capita, both for the whole US and the single regions. To do so we compute the median value in each region for every year and create a time series on R. Then we plot the whole thing in one graph:
We can see that in general, the expenditure per capita has increased from 2004 to 2013, with some ups and downs throughout the period. The downward sloping part are especially relevant in two regions, West and South between 2009 and 2010/11. We don’t have enough data, but a possible explanation could be the financial crisis which had impact on government budgeting. The largest difference between 2004 and 2013 values is observed for North-East, while the smallest is for South, of which gap between these years is of $7 circa. In the time series we only look at the median. It could be interesting to observe the same data through a boxplot to understand variability and outliers.
We start by looking at each regions and US in total.
We notice that US has a low variability, but here data for US are already considered as a total, it doesn’t consider each state observed. Instead, for the regions we capture, as before, that North-East is the one with largest variation, and we already know from the time series that this is due to the steadily increase in mh_exp per capita over the years.
The boxplots are ordered by median and we can see how North-East is the one with greatest median and how US’s median (which we can consider as the mean median across regions) is second for magnitude. Thus, it’s driven significantly by North East states expenditure.
South and Mid-West are the regions in which states seem to spend less for mental health expenditure in per capita terms.
We can clearly observe some outliers. But you can notice that they are quite clustered. Probably each group of outliers represents a state’s obervations in different years. These are not a problem for our analysis, therefore we just continue.
The second boxplot we propose is to shed the light on each region’s state.
As we expected, in regions such as South and West, where we observed outliers in the boxplot before, there are states which appear far from the others. These are District of Columbia and Alaska. The latter is indeed on the west coast, but it’s somehow detached from the other states of the region. Also District of Columbia is a case on its own since it’s not a proper state but a federal district.
We confirm that Mid-West is the region with less variability among its states in mental health expenditure per capita.
Let’s continue our univariate visualization part with demographics variables.
We do so by exploiting barplots. Again, we group results by regions as it can give us an idea of the distribution of population among the different US’s areas. Of course, we continue to look also at the total US. To group results by region we took median values and computed percentages of the population.
We start with a barplot for race composition of the population:
We immediately observe that between total US and North-East the difference is minimal. Although, no large difference is present for any of the region. In all of them there is a high prelevance of white people. The percentage for them is the highest in Mid-West area, while there’s a particular high percentage of Black/African-American population in the South.
Moreover, while the group “other race” is a minority everywhere, it is not in the West, where instead Black/African-American percentage is lower than both asian and other races.
Now on age composition:
The same results on overall observations throughout US as we had on the summary table in the data overview section return here. What’s new is the fact that we can make consideration on the “age” of each region. Although the composition of the population does not change in a relevant way.
Again, we group results by region and we took median values of the percentage of bachelor’s degree holder with age 25-44. We can notice from the following graph that, using our proxy for education, we have a lower percentage of bachelor’s holder in the South. Instead, North-East seem has 6% more educated people than the mean value of US.
We also look at a boxplot to understand the variability of education inside each region. The variability is not too high, although we observe some outliers in South, again we think they are due to District of Columbia:
Again, we group results by region we took median values and transform values in per 1000 terms. So, finally we ask ourselves the distribution of crimes in US.
South and West have the highest level of criminality, with a great departure from other regions for violent crimes and aggravated assaults. Violent crimes seem to be the most common crime, while homicide is the least frequent and it is the lowest in Mid-West.
Now that we discussed variables by themselves, we can start to see the various relationships that exist between multiple variables at the same time. Notice that when appropriate we use a log10 scale. This is useful for some of our variables because they cover a large range of values. We also decide to remove District of Columbia and United States, since in most cases the first creates outliners and is not a proper state and because US are just a total observation.
Since from the corrplot in the first part of the EDA section the correlation between mental health expenditure and criminality appeared dubious we start investigating this relationship through a scatterplot. We consider mental health expenditure per capita against the various kinds of criminality: homicides, violent crime, rape and aggravated assault.
From the scatterplot above we can see that the overall correlation is slightly negative. Which means that for an increase in public mental health per capita spending there is, on average, a decrease in criminality.
Remember that from the corrplot we have identified a positive correlation between education and mental health expenditure per capita. The higher the education the higher is the spending for health. We tried to show this through a scatterplot and the outcome is exaclty what we expected by the corrplot, even if we don’t rule out the District of Columbia.
This second scatterplot that we propose is criminality against education.
Here we can see two distinct things. First of all the correlation is negative, thus, on average, the higher the education the lower is the criminality rate. The second thing we can notice is about the log GDP. In all the criminalities, except rape, the lighter dots (higher GDP) lies above the tendency line, while the darker dots (lower GDP) lies below. Therefore, there is a positive correlation between GDP and the kind of crimes we considered, except for rape, that has a negative correlation.
Now we want to see a few of the correlation we saw before, but in the time dimension.
First of all the effect of mental health expenditure on criminality over time. We decided to report here the time series for only one crime, homicide, since the patterns are similar for all the four of them:
From this time series we can see how in the US the number of total homicides decreases over time. This can possibly follows from an increase in the mental health.
Now we check the mental health spending against the education over time.
Here we see that the increase in education over the selected decade also correspond to an increase in the public expenditure in mental health.
Finally we check the homicides against the education. Again, the pattern is similar also for other type of crimes, therefore we report only the one for homicides:
This final time series shows that in the decade of interest the decrease of homicides also correspond to an increase in education.
However to better understand all of these effect and draw stronger conclusions we should do some panel data analysis on the data-set as we will do in the next section.
Up to now we could only try to guess why such a correlation exist, and which are the social factor that induce such a result.
Such opinions for the correlations are the following: